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This  document  serves  as  the  final  technical  report  of  ONR  Contract  No.  N00014-85-K- 
0661.  The  project  continues  under  ONR  Grant  No.  N00014-90-J-1143. 

Project  Description.  The  purpose  of  the  project  was  to  develop  a  schema-based 
model  of  teaching  and  learning  and  to  test  the  adequacy  of  that  model  in  a  computer- 
based  instructional  system.  The  domain  of  instruction  is  arithmetic  word  problems,  and 
the  schemas  center  on  the  situations  that  can  be  expressed  in  such  problems. 

A  core  set  of  situations  was  identified,  and  a  series  of  studies  verified  that  the 
situations  were  sufficient  for  describing  virtually  all  legitimate  word  problems.  A  model 
of  schema  knowledge  was  constructed  for  each  of  the  basic  situations.  Each  schema 
model  specified  the  feature  knowledge,  constraint  knowledge,  planning  knowledge,  and 
implementation  knowledge  required  to  use  the  schema  successfully. 

The  instructional  system,  called  STORY  PROBLEM  SOLVER  (SPS),  was  designed 
to  provide  instruction  about  these  situations  in  such  a  way  as  to  foster  the  development 
of  appropriate  schemas  by  individuals.  The  system  consists  of  (a)  a  series  of  lessons 
requiring  about  6-8  hours  for  completion  and  (b)  a  flexible  problem-solving 
environment.  Both  of  these  components  were  designed  to  focus  on  specific  aspects  of 
schema  knowledge  required  in  solving  problems. 

In  the  lessons,  each  component  of  schema  knowledge  was  addressed  implicitly 
through  short  instructional  segments  and  related  exercises.  Students  were  introduced  to  a 
set  of  icons  depicting  the  situations,  and  they  were  encouraged  to  use  the  icons  to 
represent  the  various  situations  occurring  in  specific  problems.  A  set  of  experiments 
revealed  that  students  did  develop  the  specific  types  of  schema  knowledge  targeted  by 
SPS  and  that  the  icons  were  a  key  part  of  their  knowledge.  Moreover,  we  were  able  to 
chart  the  development  of  schemas  over  the  course  of  instruction  through  individual 
interviews  with  our  subjects. 
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The  second  part  of  the  system  is  a  flexible  problem-solving  environment  in  which 
students  can  experiment  with  problem  representations  by  manipulating  the  icons 
described  above.  Students  are  able  to  select  a  subset  of  icons  to  represent  a  problem 
and  to  link  these  together  to  represent  the  connections  in  the  problem.  They  have 
options  to  expand  the  icons  and  explore  individual  aspects  of  each  one,  to  carry  out 
calculations,  to  select  other  icons  if  they  so  desire,  or  to  have  the  system  display  a 
possible  representation  of  the  problem.  This  environment  was  developed  under  the 
original  ONR  Contract  and  evaluated  under  the  project  continuation  as  Grant  N00014- 
90-J-1143. 


Project  Results.  There  are  four  major  results.  First,  I  have  created  a  working 
computer-based  system  of  instruction  that  can  be  used  to  teach  students  about  solving 
word  problems.  The  system  has  been  used  successfully  with  about  100  subjects  to  date 
(primarily  college  students  with  weak  problem  solving  skills). 

Second,  I  have  developed  and  refined  a  theory  of  schema  structure  and 
acquisition.  The  theory  builds  on  the  general  nature  of  schema  knowledge  found  in  the 
cognitive  science  and  cognitive  psychological  literature  but  goes  considerably  beyond  it. 
In  particular,  the  theory  allows  operational  definition  of  key  components  of  a  schema 
and  thus  allows  empirical  tests  of  whether  individuals  have  acquired  these  pieces. 


Third,  as  a  direct  resuh  of  studying  the  acquisition  of  schema  knowledge  and 
attempting  to  evaluate  students’  learning,  I  have  formulated  a  new  model  of  assessment. 

The  model  is  a  network  model,  and  it  stipulates  the  need  for  assessing  both  the  number 

of  nodes  and  the  connectivity  within  the  net.  Thus,  the  project  results  allow  us  to  use 

the  theory  of  memory  organization  (i.e.,  schema  theory)  to  model  learning,  instruction, 

and  assessment.  This  result  has  had  the  most  wide  reaching  impact.  As  can  be  seen 

from  the  attached  list  of  publications  and  presentations,  I  have  been  invited  to  make  a 

number  of  contributions  about  assessing  schema  knowledge.  The  importance  here  is 

that  the  theory  developed  during  this  project  is  unique  in  its  use  of  a  common  model  for 

learning,  instruction,  and  assessment.  Moreover,  the  theory  provides  the  basis  for  a 

linkage  between  a  psychological  theory  of  memory/learning  and  a  new  psychometric  \ 

theory  of  testing.  1 


Fourth,  the  project  has  yielded  several  important  modeling  results.  We  have 
simulated  successfully  the  performance  of  students  as  they  respond  to  the  computer 
exercises.  The  simulation  uses  estimates  of  their  schema  knowledge  as  revealed  in 
interviews.  Both  correct  and  incorrect  responses  are  equally  well  estimated.  We  have 
also  employed  a  series  of  connectionist  models  which  learn  to  classify  the  situations 
expressed  in  story  problems.  The  modeling  continues  under  the  current  project  renewal. 


Attached  is  a  list  of  publications,  technical  reports,  conference  presentations,  and 

invited  addresses  that  report  research  from  this  project.  Also  listed  are  several  in-  - 

progress  activities,  including  a  book  that  describes  the  project.  ~  fiy  flats’ '■j 


Publications: 


Marshall,  Sandra  P.  (1988).  Assessing  problem  solving:  A  short-term  remedy  and  a 
long-term  solution.  In  R.  I.  Charles  &  E.  A.  Silver  (Eds.),  The  Teaching  and  Assessing  of 
Mathematical  Problem  Solving.  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 

Marshall,  Sandra  P.  (1989).  Affect  in  schema  knowledge:  Source  and  impact.  In  D.  B. 
McLeod  &  V.  M.  Adams  (Eds.),  Affect  and  mathematical  problem  solving.  New  York: 
Springer  Verlag. 

Marshall,  Sandra  P.  (1990).  The  assessment  of  schema  knowledge  for  arithmetic  story 
problems:  A  cognitive  science  perspective.  In  G.  Kulm  (Ed.),  Assessing  higher  order 
thinking  in  mathematics.  Washington,  D.C.:  AAAS. 

Marshall,  Sandra  P.  (1990).  Generating  good  items  for  diagnostic  tests.  In  N. 
Frederiksen,  R.  Glaser,  A.  Lesgold,  &  M.  Shafto  (Eds.),  Diagnostic  Monitoring  of  Skill 
and  Knowledge  Acquisition.  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 

Marshall,  Sandra  P.  (in  press).  Assessing  schema  knowledge.  In  N.  Frederiksen,  R. 
Mislevy,  &  I.  Bejar  (Eds.),  Test  Theory  for  a  New  Generation  of  Tests.  Hillsdale,  NJ: 
Lawrence  Erlbaum  Associates. 

Marshall,  Sandra  P.  (in  press).  Assessment  of  rational  number  understanding:  A 
schema-based  approach.  In  T.  Carpenter,  E.  Fennema,  &  T.  Romberg,  Rational 
Numbers:  An  Integration  of  Research.  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 


Technical  Reports: 

Marshall,  Sandra  P.,  Pribe,  Christopher  A.,  &  Smith,  Julie  D.  (1987).  Schema  Knowledge 
Structures  for  Representing  and  Understanding  Arithmetic  Story  Problems. 

Marshall,  Sandra  P.  (1988).  Assessing  Schema  Knowledge. 

Marshall,  Sandra  P.  (1988).  Schema  Knowledge  for  Solving  Arithmetic  Story  Problems: 
Some  Affective  Components. 

Marshall,  Sandra  P.,  Barthuli,  Kathryn  E.,  Brewer,  Margaret  A.,  &  Rose,  Frederic  E. 
(1989).  STORY  PROBLEM  SOLVER:  A  schema-based  system  of  instruction. 

Marshall,  Sandra  P.  (in  press).  What  Students  Leant  (and  Remember)  from  Word- 
Problem  Instruction. 

Marshall,  Sandra  P.  (in  press).  Computer-Based  Assessment  of  Schema  Knowledge  in  a 
flexible  Problem-Solving  Environment. 
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Presentations: 


Marshall,  Sandra  P.  (1987,  April).  Knowledge  representation  and  errors  of  problem 
solving:  Identifying  misconceptions.  In  W.  Montague  (Chair),  Diagnosing  Errors  in 
Science  and  Mathematics.  Symposium  conducted  at  the  Annual  Meeting  of  the 
American  Educational  Research  Association,  Washington,  D.C. 

Marshall,  Sandra  P.  (1988,  April).  Assessing  schema  knowledge.  In  N.  Frederiksen 
(Chair),  Test  Theory  for  Tests  Based  on  Cognitive  Theory.  Symposium  conducted  at  the 
Annual  Meeting  of  the  American  Educational  Research  Association,  New  Orleans. 

Marshall,  Sandra  P.  (1989,  January)  The  assessment  of  schema  kjiGw  itugc  for 
arithmetic  story  problems.  In  G.  Kulm  (Chair),  Perspectives  and  Emerging  Approaches 
for  Assessing  Higher  Order  Thinking  in  Mathematics.  Symposium  conducted  at  the  Annual 
Meeting  of  the  American  Association  for  the  Advancement  of  Science  (AAAS),  San 
Francisco. 

Marshall,  Sandra  P.  (1990,  April).  What  students  learn  (and  remember)  from  word 
problem  instruction.  In  S.  Chipman  (Chair),  Penetrating  to  the  Mathematical  Structure  of 
Word  Problems.  Symposium  conducted  at  the  Annual  Meeting  of  the  American 
Educational  Research  Association,  Boston. 


Invited  Addresses: 

"Remedial  Instruction  for  Arithmetic  Story  Problems:  A  Cognitive  Science  Approach." 
Invited  address  to  the  National  Council  of  Teachers  of  Mathematics,  Chicago,  April 
1988. 

"Schema  Knowledge".  Invited  presentation  to  the  Resource  Center  for  Science  and 
Engineering,  University  of  Puerto  Rico,  November  1989. 


Articles  in  Preparation: 

Marshall,  Sandra  P.  Understanding  the  situations  of  arithmetic  word  problems:  A  basis  for 
schema  knowledge. 

Marshall,  Sandra  P.  &  Brewer,  Margaret  A.  Learning  from  icons:  What  you  see  is  what 
you  get,  or  is  it? 


Book  in  Preparation: 

Working  title:  Schemas  in  Problem  Solving:  An  Integrated  Model  of  Instruction,  Learning, 
and  Assessment.  Negotiations  under  way  with  Cambridge  University  Press.  Contract 
expected  by  September  1991. 
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